Amplifying Actions - Towards Enactive Sound Design

Recently, artists and designers have begun to use digital technologies in order to stimulate bodily interaction, while scientists keep revealing new findings about sensorimotor contingencies, changing the way in which we understand human knowledge. However, implicit knowledge generated in artistic projects can become difficult to transfer and scientific research frequently remains isolated due to specific disciplinary languages and methodologies. By mutually enriching holistic creative approaches and highly specific scientific ways of working, this doctoral dissertation aims to set the foundation for Enactive Sound Design. It is focused on sound that engages sensorimotor experience that has been neglected within the existing design practices. The premise is that such a foundation can be best developed if grounded in transdisciplinary methods that bring together scientific and design approaches. The methodology adopted to achieve this goal is practice-based and supported by theoretical research and project analysis. Three different methodologies were formulated and evaluated during this doctoral study, based on a convergence of existing methods from design, psychology and human-computer interaction. First, a basic design approach was used to engage in a reflective creation process and to extend the existing work on interaction gestalt through hands-on activities. Second, psychophysical experiments were carried out and adapted to suit the needed shift from reception-based tests to a performance-based quantitative evaluation. Last, a set of participatory workshops were developed and conducted, within which the enactive sound exercises were iteratively tested through direct and participatory observation, questionnaires and interviews. A foundation for Enactive Sound Design developed in this dissertation includes novel methods that have been generated by extensive explorations into the fertile ground between basic design education, psychophysical experiments and participatory design. Combining creative practices with traditional task analysis further developed this basic design approach. The results were a number of abstract sonic artefacts conceptualised as the experimental apparatuses that can allow psychologists to study enactive sound experience. Furthermore, a collaboration between designers and scientists on a psychophysical study produced a new methodology for the evaluation of sensorimotor performance with tangible sound interfaces.These performance experiments have revealed that sonic feedback can support enactive learning. Finally, participatory workshops resulted in a number of novel methods focused on a holistic perspective fostered through a subjective experience of self-producing sound. They indicated the influence that such an approach may have on both artists and scientists in the future. The role of designer, as a scientific collaborator within psychological research and as a facilitator of participatory workshops, has been evaluated. Thus, this dissertation recommends a number of collaborative methods and strategies that can help designers to understand and reflectively create enactive sound objects. It is hoped that the examples of successful collaborations between designers and scientists presented in this thesis will encourage further projects and connections between different disciplines, with the final goal of creating a more engaging and a more aware sonic future.European Commission 6th Framework and European Science Foundation (COST Action

Sonification of the Human EEG

Hermann T, Baier G. Sonification of the Human EEG. In: Franinović K, Serafin S, eds. Sonic Interaction Design. Cambridge, USA: MIT Press; 2013: 285-297.This application overview summarizes the ideas, techniques, and potential application domains of the sonification of the human electroencephalogram (EEG). We start by explaining why sonification is a particularly promising tool to understand EEG data; then we review early uses of sound for the investigation of the electric brain activity, followed by a design-oriented explanation of different techniques for sonifying EEG data, many of which have been introduced by the authors. We supply sonification examples on an accompanying Web site so that the information content and aesthetics of the sonifications can be directly perceived. Finally, we give an outlook on how EEG sonification can change clinical procedures in the near future, focusing particularly on the potential of interactive sonification

Sonic Interaction Design

Serafin S, Franinović K, Hermann T, Lemaitre G, Rinott M, Rocchesso D. Sonic Interaction Design. In: Hermann T, Hunt A, Neuhoff JG, eds. The Sonification Handbook. Berlin: Logos Verlag; 2011: 87-110.This chapter introduces the emerging discipline of sonic interaction design (SID). SID is the study and exploitation of sound as one of the principal channels conveying information, meaning, and aesthetic/emotional qualities in interactive contexts. Sonic interaction design lies at the intersection of interaction design and sound and music computing